Method computer program product and device for arranging coordinate areas relative to each other

ABSTRACT

The invention relates to a method, a computer program product and a device for arranging a first and a second coordinate area relative to each other, which first and second coordinate areas are coded on a first and a second surface by a position-coding pattern. The method comprises the steps of receiving an electronic representation of a continuous line that has been drawn over the surfaces while the position-coding pattern on these was recorded, and creating an association between the first and second coordinate areas, based on a direction in which the line was drawn over a discontinuity in the Position-coding pattern, a first association being created if the line was drawn in a first direction and a second association being created if the line was drawn in a second direction.

TECHNICAL FIELD

The present invention relates to a method, a computer program productand a device for arranging a first and a second coordinate area relativeto each other.

BACKGROUND TO THE INVENTION

Various methods are available for entering information into a computer,or for “digitizing” information. Such methods comprise handling akeyboard, voice recognition, image scanning, use of the various forms ofpointing devices and recently also handwriting recognition, as is usedin many PDAs (PDA—personal digital assistant, pocket computer).

The entered information can then be stored in the user's computer or ona server that is available via a computer network, in order at a laterdate to be edited, transferred to other users, published or printed out.With the increased spread and availability of the Internet, it isincreasingly becoming possible to access electronically-storedinformation, irrespective of where the user is located. New mobiletelephone networks make possible, for example, wireless access to theInternet. As increasing amounts of information become available inelectronic form, and as physical information, that is information onpaper, becomes to an increasing extent a reflection of the electronicinformation, new needs will arise for being able to interact with theelectronically-stored information.

When entering data using any of the above methods, it can be desirableto link together various electronically-available data sets, such as twoelectronic documents or an electronic document and a meeting entry in anelectronic calendar. It could also be desirable to be able to arrangethe data sets in structures or hierarchies, in order to sort them or inorder to be able to leaf through them in a structured way using, forexample, a web-browser.

An object of the present invention is to provide a method, a computerprogram product and a device that enable a user to arrange a pluralityof such documents or data sets in a simple way.

SUMMARY OF THE INVENTION

The above object is achieved wholly or partially by a method, a computerprogram product and a device as described herein.

In one aspect, the invention is a method for arranging a first and asecond coordinate area relative to each other, which first and secondcoordinate areas are coded on a first and a second surface by aposition-coding pattern, the method comprising the steps of receiving anelectronic representation of a continuous line that has been drawn overthe surfaces while the position-coding pattern on these was recorded,and creating an association between said first and second coordinateareas, based on a direction in which the line was drawn over adiscontinuity in the position-coding pattern, a first association beingcreated if the line was drawn in a first direction and a secondassociation being created if the line was drawn in a second direction.By means of this method, associations between coordinate areas can becreated in a way that is simple and intuitive for the user. Theassociations can then be utilized in many different ways, for examplefor sorting or connecting data sets related with coordinate areas. Byusing the direction in which the line was drawn, the user can controlthe direction of the association or the relationship between thecoordinate areas. It is also possible to associate additional data setswith coordinate areas after the association has been created. Preciselyhow the direction of the line controls the relationship between thecoordinate areas can be determined by the software that utilizes orinterprets the association.

An electronic representation of a physical continuous line can be aseries of pairs of coordinates, a series of images, an image or someother machine-readable representation of a line known to those skilledin the art. A discontinuity can, for example, be caused by a jump in theposition-coding pattern by means of which the physical line istransformed_to electronic form. Such a jump can occur when the physicalline is drawn across spatially separated subsets of the position-codingpattern.

The method described above can be applied either in a user device or ina data processing unit. In addition, it can be implemented in real timeas pairs of coordinates are recorded or at a suitable time based on arepresentation of the line.

In another aspect, the invention is a computer program productcomprising a program for arranging a first and a secondelectronically-stored data set relative to each other. When executed,the program carries out the method according to the invention. Thecomputer program product can comprise a suitable storage medium and canbe executed either in the user device or in a data processing unit.

In yet another aspect, the invention is a device for arranging a firstand a second coordinate area relative to each other, which first andsecond coordinate areas are coded on a first and a second surface by aposition-coding pattern. The device has means for receiving anelectronic representation of a continuous line that has been drawn overthe surfaces while the position-coding pattern on these was recorded,and means for creating an association between said first and secondcoordinate areas, based on a direction in which the line was drawn overa discontinuity in the position-coding pattern, a first associationbeing created if the line was drawn in a first direction and a secondassociation being created if the line was drawn in a second direction.

The device according to the invention can be a user device or a dataprocessing unit, as described below.

The invention will be described in more detail below, with reference tothe accompanying schematic drawings which, for the purpose ofexemplification, show currently preferred embodiments of the invention,according to its different aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows schematically the components of a system in which thepresent invention can be used.

FIG. 2 shows schematically a user device which can be used in the systemin FIG. 1.

FIG. 3 shows schematically a position-coding pattern which can be usedin connection with the system in FIG. 1.

FIG. 4 shows schematically the placing of areas from an imaginarysurface (FIG. 4 a) on bases (FIGS. 4 b, 4 c, 4 d).

FIG. 5 shows schematically an aspect according to a first embodiment ofthe present invention.

FIG. 6 shows schematically a second embodiment of the present invention.

FIG. 7 shows schematically a third embodiment of the present invention.

FIG. 8 shows schematically a fourth embodiment of the present invention.

FIG. 9 shows schematically a fifth embodiment of the present invention.

FIG. 10 shows schematically a sixth embodiment of the present invention.

FIG. 11 shows schematically a seventh embodiment of the presentinvention.

FIG. 12 shows a schematic flow chart according to a first aspect of thepresent invention.

FIG. 13 shows a schematic flow chart according to a second aspect of thepresent invention.

FIG. 14 shows a schematic diagram of the user device's data processingunit.

DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 shows schematically components with which the invention can berealized. These comprise two or more bases, also called “surfaces”, 2 a,2 b, which are provided with a position-coding pattern P, a reading penor user device 1 that can record the position-coding pattern at the sametime as graphic marks are being made on the base. In addition, there areone or more storage units 6 for storing electronic representations ofinformation on the bases and a data processing unit 5 for processingdata stored by the storage unit and for communication with the userdevice. The storage unit 6 is shown in the accompanying drawings as onlyone storage unit, but can of course consist of several storage unitslocated in different places. In the same way, the data processing unit 5can be several data processing units, with the same or differentfunctions. In addition, the data processing unit and the storage unitcan be integrated into one unit.

The position-coding pattern on the surfaces can be recorded using theuser device 1, whereby coordinates for positions or points on a surfacecan be calculated. The coordinates can be analyzed in the user deviceand/or sent via a communication network, as discussed below, to thestorage unit or the data processing unit in order to be analyzed orstored there, or for the execution of services in response to thecoordinates recorded based on the position-coding pattern.

FIG. 2 shows a schematic example of a user device that is suitable foruse in connection with the present invention. This user device 1 can beof the type described in U.S. Pat. No. 6,674,427.

The user device 1 comprises a casing 11 which is approximately the sameshape as a pen. In the short side of the casing, there is an opening 12.The short side is intended to abut against or to be held a shortdistance from the surface or the base on which the recording of theposition-coding pattern for the position determination is to be carriedout.

The casing contains essentially an optics part, an electronic circuitrypart and a power supply.

The optics part comprises at least one light-emitting diode 13 forilluminating the surface which is to be imaged and a light-sensitivearea sensor 14, for example a CCD or CMOS sensor, for recording atwo-dimensional image. Optionally, the device can also contain anoptical system, such as a mirror and/or lens system. The light-emittingdiode can be an infrared light-emitting diode and the sensor can besensitive to infrared light.

The power supply for the device is obtained from a battery 15, which ismounted in a separate compartment in the casing. It is also possible toprovide the power supply via a cable from an external power source (notshown).

The electronic circuitry part contains a signal processor 16 whichcomprises a processor 16 a with a working memory 16 b and a programmemory 16 c. In addition, the device comprises a storage memory 16 d forstoring, for example, coordinate data. The processor 16 a is programmedto read images from the sensor, to detect the position-coding pattern inthe images and to decode this into positions in the form of pairs ofcoordinates, and to process the information thus recorded in electronicform in the way that is described in greater detail below.

In this embodiment, the device also comprises a pen point 17, by meansof which ordinary pigment-based writing can be written on the surface onwhich the position determination is to be carried out. The pen point 17can be extendable and retractable so that the user can control whetheror not it is to be used. In certain applications the device does notneed to have a pen point at all.

The pigment-based writing is suitably of such a type that it istransparent to infrared light and the position-coding pattern suitablyabsorbs infrared light. By using a light-emitting diode which emitsinfrared light and a sensor which is sensitive to infrared light, thedetection of the pattern is carried out without the above-mentionedwriting interfering with the pattern. The light-emitting diode and thesensor is triggered synchronously by the signal processor 16, suitablyat a fixed frequency, typically about 50-100 Hz.

In addition, the device can comprise buttons 18, by means of which thedevice can be activated and controlled. It can also have a transceiver19 for wireless transmission, for example using infrared light, radiowaves or ultrasound, of information to and from the device. The devicecan also comprise a display 20 for displaying positions or recordedinformation.

The device can be divided between different physical casings, a firstcasing containing components which are required for recording images ofthe position-coding pattern and for transmitting these to componentswhich are contained in a second casing and which carry out the positiondetermination on the basis of the recorded image(s).

FIG. 3 shows a schematic example of a position-coding pattern P that canbe used in connection with the present invention and that can bearranged, for example, on the bases 2 a and 2 b. In the otheraccompanying drawings, the position-coding pattern P is shown onlyschematically, as a surface provided with dots.

This position-coding pattern is used to record in electronic form whatis written on the base. Different types of position-coding pattern, thatcan be used for this purpose, are previously known. For example, U.S.Pat. No. 5,477,012 shows a position-coding pattern where each positionis coded by a unique symbol. The position-coding pattern can be read offusing a pen, which detects the position code optically, decodes this andgenerates pairs of coordinates which describe the movement of the penacross the surface. In U.S. Pat. Nos. 6,570,104 and 6,663,008, which areincorporated herein by reference and are both assigned to the Applicantof the present application, another position-coding pattern isdescribed, where each position is coded using a plurality of symbols ofa simpler type and where each symbol contributes to the coding of morethan one position. In U.S. Pat. No. 6,570,104, different sized dots areused to code ones and zeros in the position-coding pattern, which isbinary. In U.S. Pat. No. 6,663,008, four different displacements of adot from a nominal position are used to code four different bit pairs inthe position-coding pattern. A certain number of dots, for example 6*6dots, P1, P2, codes a unique position. The position can be calculatedfrom the bit values corresponding to the dots.

The position-coding patterns in U.S. Pat. Nos. 6,570,104 and 6,663,008can be detected optically using a pen that decodes the dots andgenerates a pair of coordinates for each set of, for example, 6*6 dots.If the position-coding pattern is read off while the pen is writing onthe position-coding pattern, a sequence or series of pairs ofcoordinates is thus obtained, which sequence or series describes themovement of the pen across the position-coding pattern and thusconstitutes an electronic representation of what is being written on thebase.

In the following, it is assumed that the base 2 a and 2 b is providedwith a position-coding pattern of the type that is described in, forexample, U.S. Pat. No. 6,663,008.

The position-coding pattern in U.S. Pat. No. 6,663,008 can codecoordinates for a very large number of points or positions. These pointscan together be said to form an imaginary surface, that is a “virtual”or “theoretical” surface, which can be described using coordinates, butwhich does not actually exist in the physical sense of the word. FIG. 4a shows schematically such an imaginary surface 4, on which three areasA, B and C have been marked.

The imaginary surface can be divided into smaller areas by pairs ofcoordinates that represent the limits of each such area being stored in,for example, a database. On the imaginary surface in FIG. 4 a, the threeareas A, B and C are delimited by pairs of coordinates that representthe corner points of the respective areas.

By means of a position-coding pattern as described above, suchcoordinates can be coded in machine-readable form. By recording theposition-coding pattern at a point, the coordinates of the point on theimaginary surface can be calculated.

By arranging the position-coding pattern on a base 2, for example asheet of paper, coordinates can be coded on the base 2 for points on theimaginary surface 4. If an area on the imaginary surface is coded by aposition-coding pattern arranged on a base, which position-codingpattern is reserved for precisely that base, it can be said that thearea has been “cut out” from the imaginary surface and placed on thebase 2. This is described in greater detail in for example U.S.Publication Nos. 2003/0061188, 2003/0046256, and 2002/0091711, which areincorporated herein by reference. The imaginary surface can be dividedinto areas and subareas by the areas being specified in a database andtherein linked with, for example, functions or owners. When an area isallocated to a function or an owner, it can be reserved so that itcannot at the same time be allocated to other functions or owners. Alist of reserved areas can be made available, for example via theInternet.

FIG. 4 b shows a first example of this, where the areas A and B from theimaginary surface 4 in FIG. 4 have been “cut out” and arranged next toeach other on a base 2, by the coordinate areas that code correspondingareas on the imaginary surface being arranged on the base.

FIG. 4 c shows how the areas A and C have been “cut out” from theimaginary surface 4 and arranged next to each other on a base 2, in thesame way as in FIG. 4 b.

FIG. 4 d shows how the cut-out areas A and C are arranged on differentbases 2 a, 2 b and how the bases are then brought together in order thata line can be drawn across the areas A and C, whereby recording of theposition-coding pattern can be carried out using the user device.

FIGS. 4 b and 4 c thus show bases each with two coordinate areas, A, Band A, C respectively.

A function can be associated with each area. The coordinates, forexample coordinates of two diagonally opposite corner points of therespective area, are then stored in a data base, so that, given a pairof coordinates in the database, it is possible to locate the area towhich the pair of coordinates belongs. In this database respective areascan then be linked with, for example, addresses to which data that ismarked in an area is to be sent, applications, functions or other areasthat are associated with or linked to the area.

It is also possible to define in the database subscribers,license-holders or owners of areas, so that information that is markedin a particular area can be linked to or sent to a predetermined unit.An example of entries in such a database is shown in Table 1.

TABLE 1 Example of entries in a database containing n areas. AreaBelongs to Function X₁, Y₁; X₂, Y₂ Company 1 Send.to (111.11.11.1) X₃,Y₃; X₄, Y₄ Company 2 Send.to (222.22.22.2) . . . . . . X_(n−1), Y_(n−1);X_(n), Y_(n) Company n Send.to (nnn.nn.nn.n)

When, for example, a pair of coordinates in an area according to thesecond row in Table 1 is recorded by the user device or is received in aserver, the data that was entered in the area in question or on the basein question can be sent to an IP address (in the example according toTable 1: 111.11.11.1), that belongs to Company 1, for furtherprocessing.

A database as described above can be found both in the pen 1, for thefunctions that the pen 1 is to recognize directly, or in the storageunit 6 or in association with the data processing unit 5, for thefunctions that can be carried out from there. As coordinates that codethe respective areas are arranged on bases, as shown in FIGS. 4 b and 4c, the areas that are associated with functions or network addresses canbe defined on the bases. When pairs of coordinates within an area arerecorded by a user device 1, this can cause a function to be executed inthe user device or in some unit with which the user device 1communicates.

Each area consists in the respective dimension (x,y) of a quantity ofsequential pairs of coordinates, for which reason a line that was drawnand recorded within an area (for example the area A+B in FIG. 4 b) usingthe user device will comprise a series of pairs of coordinates. By aline is meant here a graphic mark that was made by a user device on abase without raising the user device from the base. The fact that theuser device has been raised from the base can be detected, for example,by the pen point of the user device being connected to a pressuresensor, or by the distance to the position-coding pattern being measuredoptically.

A line that is continuous on a physical surface can be interpreted inseveral different ways, in addition to as an ordinary graphic line. Byphysically continuous is meant that the line has been made with onestroke of the user device. The electronic representation of the linecan, however, be continuous or discontinuous depending upon theposition-coding pattern on the base on which the line was recorded. Bythe electronic representation of the line being continuous is meant thatthe distance on the imaginary surface between pairs of coordinates fortwo points recorded directly after each other is less than a certainpredetermined distance value. A physically continuous line can thus bediscontinuous if, when it is recorded in electronic form, it containspairs of coordinates from at least two different, spatially separateareas of the position-coding pattern.

If a line is thus drawn and recorded across two areas (for example theareas A and C in FIG. 4 c), which are not adjacent to each other on theimaginary surface, the electronically recorded line will contain adiscontinuity. A discontinuity can, for example, be detected when thedistance on the imaginary surface between two points recorded directlyafter each other is larger than said predetermined distance value. Thepredetermined distance value can be selected depending upon which typeof area is being recorded. An electronically recorded line comprisingsuch a discontinuity is in the following called a “hyperline”.

Discontinuities can arise either as a result of two areas of theposition-coding pattern having been arranged on the same base (FIG. 4 c)or when two bases on which different areas of the position-codingpattern are arranged (FIG. 4 d) are brought together so that a line canbe drawn across them.

The hyperline can be given a signification that relates to arelationship between the different areas across which the line is drawn.This signification can be to connect logically or “associate” the areaswith each other, so that for example one data set associated with afirst area is associated with a data set that belongs to a second area.Such an association can be stored in different ways in a storage unitand thereby utilized by applications that have access to the storageunit.

The signification can, as discussed below, be dependent upon thedirection of the drawn line, so that for example a note page and abusiness card are associated with each other in a way that is dependentupon the direction in which the line was drawn. The associations can behandled in the same way as the above-mentioned associations of areaswith owners and functions, as described in connection with Table 1above. This will be discussed in greater detail below.

FIG. 1 shows two bases 2 a, 2 b, which are wholly or partially providedwith a position-coding pattern P. The bases are preferably connected toinformation which has been digitized and is stored and accessible inelectronic form 6 a, 6 b. The information may be connected to the base 2a, 2 b by a user writing it down on an area that is provided with aposition-coding pattern P, and/or by it being created electronically andprinted on the base 2 a, 2 b. The bases may also be connected toadditional electronic information which is not written or printedthereon. The electronic form of the information can be stored aselectronic documents such as .pdf (Adobe Acrobat®), .doc (MS Word®),.svf (standard vector format) or other formats, but it is also possibleto store the information as some form of raw data such as a series ofpoints, coordinates or vectors which represents information that waswritten down on the base using a user device. More or less processedversions of such raw data, such as text files from OCR software, canalso be stored and utilized.

The storage can, for example, be in a personal computer, a PDA, a mobiletelephone or simply in the actual user device. A server comprising adata processing unit 5 and a storage unit 6 can also be used. Differentelectronic documents 6 a, 6 b can also be stored in different storageunits 6, even though this is not shown in FIG. 1.

In addition, a communication means 7 can be arranged between the userdevice 1 and the storage unit 6. This communication means 7 can, forexample, be a computer, a mobile telephone, a mobile telephone network,a telephone network, a computer network or a combination of these.Precisely how the communication means operates is of sub-ordinatesignificance, as its function is to make possible the transmission ofdata from the user device to the storage unit, in a way known to thoseskilled in the art. It is also possible to connect the user devicedirectly to a computer network via cable, short-range radio link,infrared link, mobile telephone network or in some other suitable way.

Optionally, the storage unit 6 can also be connected to a dataprocessing unit 5, such as a personal computer or some other suitabledevice equipped with a processor. The data processing unit 5 can bearranged to read electronic documents 6 a, 6 b stored in the storageunit and can be provided with software for analyzing the documents orthe files, and for taking measures based on the result of such analyses.The data processing unit 5 can also act as an intermediary between thestorage unit 6 and the communication means 7, and can also communicatedirectly with the user device 1 and other units. In this way, the dataprocessing unit 5 can receive, analyze and convert information going toand from the user device 1.

FIG. 5, like FIG. 1, shows a first embodiment of the invention, wheretwo bases 2 a, 2 b are utilized, which contain information that is alsostored in electronic form 6 a, 6 b in a storage unit 6 and which areprovided with a position-coding pattern P. The information can, asdescribed above, be pre-printed on the base 2 a, 2 b or written down bythe user. When this was carried out is less important, provided that theinformation is stored and available in electronic form in the storageunit 6.

FIGS. 5-10 show a broken line that schematically separates the physicalaspects of the invention (use of the user device and the base) from itslogical aspects (processing in the data processing unit/storage unit).

The user now wants to create a reference 3 a from a first document 2 ato a second document 2 b, so that when the user or some other personreads the corresponding first electronic document 6 a via his/hercomputer, it will automatically contain a reference 3 a to thecorresponding second electronic document 6 b.

According to the invention, the user places both the bases 2 a, 2 b nextto or overlapping each other. The user draws a line 3 with his userdevice 1 from the first base 2 a, across the transition between thebases and a little way onto the second base 2 b.

When the user passes the user device across the bases, the user device 1records a series of pairs of coordinates, which corresponds to the line3. The pairs of coordinates are usually stored as raw data in a memoryin the user device and are not sent to the storage unit/data processingunit 6/5 until the user device 1 has recorded an area that indicatesthat sending is to be initiated.

In the transition between the bases 2 a, 2 b, the user device 1 records,however, a discontinuity, since a recorded pair of coordinates does notbelong to the same area as the preceding pair of coordinates. When thediscontinuity is recorded, the user device 1 first investigates whetherany one of the areas belongs to a predetermined function area, forexample a “send” area. If such is the case, this function can be carriedout, for example, sending the data that is stored in the storage memory16 d of the user device (FIG. 2). If none of the areas is a functionarea, the hyperline 3 can instead be recorded as an association 3 a,that is an association between the areas, as described above. Theassociation can be stored in the storage memory of the user device andcan comprise references to the areas between which the line 3 was drawn.As an alternative, the series of pairs of coordinates can be storedprecisely as the other pairs of coordinates, to be analyzed later in thedata processing unit 5, whereby detection of the discontinuity iscarried out in the data processing unit.

When the user then marks an area that has a “send”function, the pairs ofcoordinates stored in the memory are sent, along with any associations,via the computer network 7 to the storage unit 6. Optionally, only datathat is associated with the “send” function, the surface in question orsome recorded association can be sent.

When the pairs of coordinates are sent to the storage unit 6, they caneither be stored as they are for later processing, or can be processedimmediately by the data processing unit 5. Such processing can compriseinterpretation of recorded data by for example OCR or ICR. Theprocessing can also comprise conversion to other storage formats, suchas text document, graphics files, forms, spreadsheets or databaseentries. The processing can also comprise analysis and storage ofrecorded associations 3 a.

As shown in FIG. 5, the associations 3 a recorded by the user device 1can be analyzed with regard to which areas the line has been drawnacross. For example, a “to” area and a “from” area can be determined,based on the time sequence or order in which the coordinates wererecorded on the respective side of the discontinuity.

FIGS. 5 a and 5 b show how an association between two bases 2 a and 2 bcan differ depending upon the direction in which the line 3 was drawnwith the user device 1. In FIG. 5 a, the base 2 a was recorded beforethe base 2 b. The association can thus be that the electronic document 6a refers to the electronic document 6 b. In FIG. 5 b, on the other hand,the base 2 b was recorded before the base 2 a, for which reason theassociation can be that the electronic document 6 b refers to theelectronic document 6 a.

The associations can be stored in the storage unit 6. The storage of theassociation 3 a can be carried out in various ways. According to apossible embodiment, associations are stored in a database in the formof relationships between areas, as shown in Table 2 below. In Table 2,Area 1 can, for example, be defined as a “from” area and Area 2 as a“to” area, where the “from”area can be the area that was recorded firstwhen the line was drawn and the “to” area can be the area that wasrecorded last when the line was drawn. This information can, forexample, be utilized in such a way that Area 1 refers to Area 2, withoutArea 2 necessarily referring to Area 1. It is, of course, also possibleto invert the definition of “from” area and “to” area, so that Area 2refers to Area 1. Which area is defined as the “to” area and the “from”area can also be application-specific, that is can depend on thesoftware that receives, utilizes and/or interprets the association.

TABLE 2 Example of database of associations. Area 1 Area 2 X₁, Y₁; X₂,Y₂ X₃, Y₃; X₄, Y₄ X₅, Y₅; X₆, Y₆ X₇, Y₇; X₈, Y₈ . . . . . . X_(n−3),Y_(n−3); X_(n−2), Y_(n−2) X_(n−1), Y_(n−1); X_(n), Y_(n)

It is also possible to store associations in other forms than the setsof coordinates shown in Table 2, for example as document name, pagenumber or other designations designed for the specific areas. It is alsopossible to divide the imaginary surface into subareas with, forexample, a similar function, use or owner. In this way, precise pairs ofcoordinates do not necessarily need to be stated in the associationdatabase in Table 2 in order for an association to be able to work, butonly a sufficiently large part of the pairs of coordinates to enable thearea to be identified with sufficient accuracy for the application inquestion. The associations can also be converted to designations thatrefer directly to the electronic documents that are associated, forexample hyperlinks or search paths in a storage unit or in a computernetwork. The storage of the associations can be carried out inelectronic form in a storage medium suitable for the purpose.

FIG. 6 shows a second embodiment of the present invention, in which abusiness card 30 and a calendar 31 are associated with each other. As inthe first embodiment, it is assumed here that the business card 30 andthe calendar 31 are provided with a position-coding pattern P.Similarly, electronic equivalents 30 a, 31 a of the business card andthe calendar respectively are stored and available.

The business card 30 and the calendar 31 or the calendar page are placedin such a way that a line 3 can be drawn from one to the other. Theassociation 3 a that is hereby created can be made to depend on theorder in which the business card 30 and the calendar page 31 wererecorded.

If the position-coding pattern P on the business card 30 was recordedfirst and then the position-coding pattern on the calendar page 31, thiscan mean that the information on the business card 30 is entered ontothe calendar 31 and/or that a meeting is arranged with the owner of thebusiness card.

If, on the other hand, the calendar page 31 is recorded before thebusiness card 30, this can be interpreted that a message is to be sentto the owner of the business card, which message can include aninvitation to a meeting. Optionally, the time and date of the meetingcan be determined by where the line starts or ends on the calendar page31.

FIG. 7 shows a third embodiment of the present invention, where a base32 with notes is associated with a calendar page 31. Depending upon thedirection in which the line 3 is drawn that associates the bases witheach other, different effects can be achieved.

If, for example, the base with notes 32 is recorded first, this can meanthat the notes are associated with a particular meeting or a particularcalendar time. On the basis of this association 3 a, a meeting entry in,for example, Microsoft Outlook® can be created or modified so that thenotes 32 a are attached to the meeting entry 31 a.

If instead the calendar page 31 is recorded first, this can mean thatthe content of this, such as notes, date or the like, is transferred tothe notes 32 a, or that the note page 32 a is dated according to thecalendar page 31 a. It can also mean that data from the calendar page 31a, such as time, place and participants in a meeting, is transferred tothe notes 32 a.

FIG. 8 shows a fourth embodiment, in which a surface 34, for example adesktop, a refrigerator or a notice board, is provided with aposition-coding pattern P and a number of bases 33 containinginformation are placed on the surface. The bases 33 are also providedwith position-coding patterns. By drawing hyperlines 3 from therespective bases 33 to the physical surface 34, electronic equivalentsof the content on the bases 33 a are associated with an electronicversion 34 a of the physical surface 34, for example the “Desktop” inWindows®.

If instead the physical surface 34 is a refrigerator, memo notes 33 thatare stuck on the refrigerator can, for example, be associated with anelectronic version of the refrigerator surface and can be stored therefor transmission to, for example, a calendar, a “to-do”list, a shoppinglist or an electronic message.

FIG. 9 shows a fifth embodiment of the present invention, in which asequence order between a number of bases 35 is recorded. The bases areprovided with a position-coding pattern and can be print-outs of, forexample, pages in a book, photographs in an album or OHP slides for apresentation.

The user arranges the images in the sequence order in which he/she wantsthem and then draws lines 3 between pairs of bases 35, or across all thebases in a single line. By recording the direction of the lines 3, thebases are associated with each other and a sequence order can bedistinguished, after which the electronically-stored equivalents 35 a ofthe bases can be sorted and displayed.

FIG. 10 shows an extension of the embodiment that was described inconnection with FIG. 3, where documents 36 are associated with eachother by means of lines 3 that indicate references from one document 36to another 36. By recording the direction of the lines 3 that are drawnfrom document to document, it is possible to determine how the documents36, 36 a relate to each other, so that a hierarchy can be created. Forexample, this can be used to create a structure of electronic documents36 a, where, using print-outs of the documents 36, references arecreated between them by placing the documents on a surface and thendrawing lines 3 between them in the order that is required. Thereferences can advantageously be incorporated in the electronicdocuments in the form of hyperlinks, but they can also be stored asdescribed above, in the form of database entries. It is also possible toutilize associations to create literature references to other documents.

FIG. 11 shows an embodiment of the invention in which notes that werewritten down on a base 32 and recorded in electronic form 32 a are sentas an e-mail message 38, by a line 3 being drawn from the base 32 withthe notes to a business card 30 that is provided with a position-codingpattern, and are sent to the owner of the business card as, for example,an e-mail or fax message. The e-mail message is received at a secondstorage unit 37 where it may be accessed by the recipient, i.e. theowner of the business card. This can be made possible by informationcorresponding to the business card being stored in a storage unit 6.When the line is drawn from the base 32 to the business card, anassociation is created between the base 32 and the business card 30.

By the line 3 being drawn instead from the business card 30 to the base32 with the notes, a second function can be initiated, for exampleinformation from the business card being added to the notes.

FIG. 12 shows a flow chart for the entry of data into the pen 1. Whenthe user device is passed in a line across a base in step 40, itreceives image data in step 41 in the form of a series of images thatrepresents the position-coding pattern on the base. In addition, in step41 a, time data is received, that indicates when the respective imagesin the series were taken. The time resolution of the time data ismatched advantageously to the interval between the images, for example1/50 or 1/100 of a second. In this connection, the time data can berelated to the order in which pairs of coordinates are recorded, ratherthan the actual times at which the recording is carried out. The imagedata is analyzed in order to calculate, in step 41 b, pairs ofcoordinates for points on the line on the basis of the position-codingpattern. The processing of image data and time data can be carried outin real time, for example so that a pair of coordinates is calculatedfor each image, as soon as this has been recorded. The flow chart inFIG. 12 can thus be interpreted in such a way that the step 41 ofreceiving image data means that an image is received and that the step41 b of identifying pairs of coordinates means calculating a pair ofcoordinates in the image, etc. The calculated pairs of coordinates forma series of pairs of coordinates, which can constitute an electronicrepresentation of the line. FIG. 12 can, however, also be interpreted insuch a way that a complete sequence of image data is received in step41. This method can be utilized when the identification does not need tobe carried out in real time, such as in the data processing unit 5. Theinterpretation of FIG. 12 can thus be governed by whether the method isto be implemented in a user device 1 or in a data processing unit 5.

In step 42, the pairs of coordinates are stored in a storage memory 16d, 48. The stored pairs of coordinates can be provided with time stamps,that give the precise time at which each pair of coordinates wasrecorded. As the pairs of coordinates are recorded, an analysis of themcan be carried out, whereby discontinuities are detected in step 43.When a discontinuity is detected, this can be analyzed in order todetermine in step 44 whether the discontinuity leads to an area with apre-determined function, such as for example a “send” area. If such isthe case, the stored pairs of coordinates associated with the “send”function are sent in step 46 to a storage unit/data processing unit forfurther processing. If the discontinuity does not lead to a functionarea, the user device can create a flag in step 45 that indicates thediscontinuity. Such a flag can have various forms and, in step 47, canadvantageously be stored in the user device's memory 16 b, 16 c or inthe storage memory 16 d to be sent later to the data processing unit 5or the storage unit 6. The flag can in practice be interpreted by thedata processing unit as an association between the indicated coordinateareas. For example, the flag can contain “from” area and “to” area insome suitable form, or it can contain pairs of coordinates for therespective sides of the discontinuity and time stamps for the respectivepairs of coordinates. It is also possible to store the discontinuity instep 47 in the form of a hyperlink or as a combination of the above. Asmentioned above, the step of creating a flag is not necessary, as theseries of pairs of coordinates can be analyzed afterwards instead. Thestorage of data in the user device or in the storage unit 6 canadvantageously be preceded by compression in a way known to thoseskilled in the art.

FIG. 13 shows reception and processing in the data processing unit 5.According to a preferred embodiment of the invention, in step 50 thedata processing unit receives data from the user device in the form ofseries of pairs of coordinates or images, which represents an electronicversion of the movement of the user device across the imaginary surfacethat is coded by the position-coding pattern. In step 51, received data52 can be stored in the storage unit 6 before or in conjunction with thedata being processed by the data processing unit. Identifyingdiscontinuities in step 53 can be carried out in the same way as in theuser device, that is by analysis of the distance between pairs ofcoordinates that were recorded close to each other in time. It is,however, also possible to identify the discontinuities on the basis ofinformation that is provided by the user device, such as flags, etc,which indicate discontinuities that have already been identified by theuser device. The analysis in step 54 of the discontinuity can be carriedout completely in the user device, completely in the data processingunit or can be shared between them, so that a part of the analysis iscarried out in the user device 1 and the remainder in the dataprocessing unit 5.

The discontinuity is analyzed, among other things, with the aim ofdetermining in step 55 whether any of the areas is associated with afunction, for example sending an e-mail message or fax message. If suchis the case, in step 56, this function is executed in a way known tothose skilled in the art. If, on the other hand, the discontinuity isnot associated with a function, it can be assumed to be an association,whereupon association data 58, as described above, is created in step 57and is stored is a suitable storage unit 6.

FIG. 14 shows a schematic diagram of the data processing structure ofthe user device 1, which data processing structure comprises a processor16 a which is arranged to receive input data, analyze it and generateoutput data for, for example, working memory 16 b, program memory 16 c,storage memory 16 d (FIG. 2) or for communication with external units 5,6 (FIG. 1). The data processing structure according to a preferredembodiment comprises input means 21 comprising means 21 a for receivingimage data and means 21 b for receiving time data from atime-measurement means 24. The data that is received is analyzed in ananalysis means 22 and is output by output means 23. The above-mentionedmeans 21, 22, 23, 24 can consist of customer-specific integratedcircuits adapted for the purpose (for example ASIC) or of some form ofprogrammable integrated circuit (for example PROM, FPGA) or an ordinaryprocessor with software.

ALTERNATIVE EMBODIMENTS

It is possible only to record pairs of coordinates in the user device,which pairs of coordinates are stored in the user device's storagememory until the user device records a “send” function. When the “send”function is recorded, the pairs of coordinates in the user device'smemory are sent to the data processing unit/storage unit for furtherprocessing. Alternatively, the user device may operate to continuouslytransmit or “stream” the recorded pairs of coordinates, or the recordedimages of the positions-coding pattern, to the data processingunit/storage unit for further processing.

If the data sets that are sent to the data processing unit/storage unitare grouped according to which area of the imaginary surface they wereentered on, the data processing unit will not necessarily be able tofind two consecutive pairs of coordinates which belong to differentcoordinate areas. This can also occur if the pairs of coordinates aregrouped side by side, on the basis of which base they were recorded on.

The data processing unit can then instead analyze the receivedcoordinates and identify the lines that could constitute hyperlines.Examples of such are lines that lie at an outer edge of a coordinatearea. These lines can then be analyzed in the way described above, thatis with regard to the time stamps of the pairs of coordinates that formthe line.

It is also possible to analyze each line individually, on the basis ofthe time stamp of each pair of coordinates. This can be carried out by“following”, in a particular coordinate area, each line from itsbeginning to its end and by also analyzing, in other coordinate areas,lines that lie close in time to the start point or end point of theanalyzed line.

An alternative method for identifying discontinuities is to storeinformation in a memory regarding where the imaginary surface has beendivided or “cut out”, that is where the borders lie between thedifferent areas. Using this information, lines that are recorded can beinvestigated to see whether they cross a predetermined border betweendifferent coordinate areas.

Yet another way of identifying discontinuities is to utilize aposition-coding pattern that can code coordinate areas or subareasdirectly. In this way, it is possible to calculate the area affiliationof a pair of coordinates directly from the position-coding pattern. Adifference in the area affiliation between two pairs of coordinatesrecorded directly after each other in time can thereby be an indicationof a discontinuity.

Another way of storing representations of the line from whichassociations can be derived, is in the form of files in image or vectorformat, that represent the series of pairs of coordinates that wererecorded by the user device 1 when lines were drawn on a base providedwith a position-coding pattern. Such a file can comprise lines that aredrawn over any areas from the whole of the imaginary surface, or onlylines that are drawn within certain subareas, for example subareas thatare associated with a specific base, such as a page of a document or thelike. By the use of time data for each pair of coordinates in such aseries, it is possible to scan or follow lines in such files inchronological order. By means of such a method, it is possible to findjumps on the imaginary surface between two pairs of coordinates thatwere recorded directly after each other. Alternatively, a series ofpairs of coordinates can be analyzed in order to determine whether itcomprises more than one coordinate area. Optionally, such a file inimage or vector format can be used as the sole storage source forassociations.

It is possible to store in a database “keys” to each recorded line,where the key indicates, for example, a start coordinate or endcoordinate for each recorded line. By means of the keys, the file inimage or vector format can be scanned line by line, which can save timein comparison with scanning the whole file in chronological order.Optionally, knowledge of the approximate recording time or areaaffiliation can be used as supplementary input data.

All the ways of storing or representing associations and lines can beused in the user device 1 or in the data processing unit/storage unit5/6. The choice between them can be determined by factors such asprocessor capacity, search speed or storage capacity.

As a further alternative, associations between documents can be storedin or directly associated with those documents. An example of suchstorage is to provide a document, that refers to another document, witha hyperlink or the like. Such hyperlinks can be inserted whereappropriate, for example directly in the text, in foot notes or in endnotes/literature references.

Another alternative is to let the association govern the physicalstorage of associated documents, so that each document is stored in aplace in the search path hierarchy, which place is determined by thedocuments associated with the document.

The invention can also be used on bases that have been provided withposition-coding patterns, but have not yet been provided withinformation. As a result, “pre-determined associations” can be created,which associate areas on the imaginary surface with each other, so thatthe information that is later written on the base or is associated withthe respective areas in some other way, will be included in theassociation.

The embodiments described above are to be regarded as examples ofapplications of the present invention. It is, of course, possible tocombine the described applications or parts thereof in a plurality ofways. In addition, a very large number of other applications arepossible and are covered by the appended claims.

1. A method for associating a first coordinate area on a first surfacewith a second coordinate area on a second surface, said first and secondcoordinate areas being coded by a position-coding pattern, said methodcomprising: receiving an electronic representation of a continuous linethat has been drawn from either the first surface or the second surfaceto the other surface; and creating an association between said first andsecond coordinate areas based on a direction in which the line was drawnover a discontinuity in the position-coding pattern between the firstsurface and the second surface, a first association being created if theline was drawn in a first direction and a second association beingcreated if the line was drawn in a second direction.
 2. The method asclaimed in claim 1, wherein the step of receiving an electronicrepresentation comprises receiving a series of pairs of coordinates thatrepresents the line and that has been determined from theposition-coding pattern.
 3. The method as claimed in claim 1, furthercomprising: detecting the discontinuity by comparing a distance betweentwo pairs of coordinates which were recorded during a predeterminedinterval of time with a predetermined distance value.
 4. The method asclaimed in claim 1, further comprising: receiving in electronic form anindication of the discontinuity in the position-coding pattern.
 5. Themethod as claimed in claim 1, wherein the step of creating anassociation comprises determining the direction of the line by comparinga time of recording a first pair of coordinates included in the linewith a time of recording a second pair of coordinates included in theline, which pairs of coordinates lie on separate sides of thediscontinuity.
 6. The method as claimed in claim 1, wherein the step ofreceiving an electronic representation comprises receiving in electronicform an indication of the direction.
 7. The method as claimed in claim1, wherein the step of creating an association comprises creating adatabase entry comprising a reference to said first and secondcoordinate areas and an indication of the direction between them.
 8. Themethod as claimed in claim 1, wherein which the step of receiving anelectronic representation comprises receiving the electronicrepresentation from a user device.
 9. The method as claimed in claim 1,wherein the step of receiving an electronic representation comprisesreceiving the electronic representation from a sensor.
 10. The method asclaimed in claim 1, wherein said first and second coordinate areas areassociated with first and second data sets respectively.
 11. The methodas claimed in claim 10, wherein said data sets comprise electronicdocuments.
 12. The method as claimed in claim 10, wherein saidassociation is a reference from one of said data sets to the other. 13.The method as claimed in claim 10, wherein the step of creating anassociation comprises creating a database entry that has a reference tosaid first and second data sets and an indication of the directionbetween them.
 14. The method as claimed in claim 10, wherein the step ofcreating an association comprises modifying one of said data sets tocomprise a reference to the other of said data sets.
 15. The method asclaimed in claim 14, wherein said reference is a hyperlink.
 16. Acomputer program product comprising a program, tangibly embodied on acomputer readable storage medium, for arranging a first and a secondelectronically-stored data set relative to each other wherein, whenexecuted, the program carries out the method according to claim
 1. 17. Adevice for associating a first coordinate area on a first surface with asecond coordinate area on a second surface, said first and secondcoordinate areas being coded by a position-coding pattern, said devicecomprising: means for receiving an electronic representation of acontinuous line that has been drawn from either the first surface or thesecond surface to the other surface; and means for creating anassociation between said first and second coordinate areas based on adirection in which the line was drawn over a discontinuity in theposition-coding pattern between the first surface and the secondsurface, a first association being created if the line was drawn in afirst direction and a second association being created if the line wasdrawn in a second direction.
 18. The device as claimed in claim 17,further comprising: a sensor for reading off the position-coding patternon said first and second surfaces.
 19. The device as claimed in claim17, wherein the device is a user device which is configured fordigitizing handwriting.
 20. The device as claimed in claim 17, furthercomprising: means for communication with a user device.